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Abstract. We answer a question of L. Grasedyck that arose in quantum information theory, 
showing that the Hmit of tensors in a space of tensor network states need not be a tensor network 
state. We also give geometric descriptions of spaces of tensor networks states corresponding to 
trees and loops. Grasedyck's question has a surprising connection to the area of Geometric 
Complexity Theory, in that the result is equivalent to the statement that the boundary of 
the Mulmuley-Sohoni type variety associated to matrix multiplication is strictly larger than 
the projections of matrix multiplication (and re-expressions of matrix multiplication and its 
projections after changes of bases). Tensor Network States are also related to graphical models 
in algebraic statistics. 



1. Introduction 

1.1. Origin in physics. Tensors describe states of quantum mechanical systems. If a system 
has n particles, its state is an element of Hi0 • • • (8) Hn with Hj Hilbert spaces. In numerical 
many-body physics, in particular solid state physics, one wants to simulate quantum states of 
thousands of particles, often arranged on a regular lattice (e.g., atoms in a crystal). Due to the 
exponential growth of the dimension of i^i® • • • (8) Hn with n, any nai've method of representing 
these tensors is intractable on a computer. Tensor network states were defined to reduce the 
complexity of the spaces involved by restricting to a subset of tensors that is physically reason- 
able, in the sense that the corresponding spaces of tensors are only locally entangled because 
interactions (entanglement) in the physical world appear to just happen locally. 

Such spaces have been studied since the 1980's. These spaces are associated to graphs, and 
go under different names: tensor network states, finitely correlated states (FCS), valence-bond 
solids (VBS), matrix product states (MPS), projected entangled pairs states (PEPS), and multi- 
scale entanglement renormalization ansatz states (MERA), see, e.g., |141 [71 [9} [6l 1151 [5] and the 
references therein. We will use the term tensor network states. 

1.2. Definitions and notation. For a graph V with edges Cs and vertices vj, s & e{j) means Cg 
is incident to Vj. If F is directed, s G in{j) are the incoming edges and s G out{j) the outgoing 
edges. 

Let Vi, . . . ,Vn be complex vector spaces, let Vj = dim Vj. Let F be a graph with n vertices 
Vj, 1 < j < n, and m edges e^, 1 < s < m, and let e = (ei, . . . ,6^) G N™'. Associate Vj to 
the vertex Vj and an auxiliary vector space Eg of dimension to the edge e^. Make F into a 
directed graph. (The choice of directions will not effect the end result.) Let V = Vi® • • • (8) V^. 

Let 

(1) riV5(F,e,V) := 

{T G V I 3Tj G Vj0{(S)s^in(j)Es)0{(^teout{j)Et), such that T = Con{Ti® • • • (8 r„)} 
where Con is the contraction of all the E^^s with all the EVs. 
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Example 1.2.1. Let T be a graph with two vertices and one edge connecting them, then, 
TNS(r, ei, Vi(8>V2) is just the set of elements of Vi(8)V2 of rank at most ei, denoted {SegiFVi x 
Pl^)) and cahed the (cone over the) ei-st secant variety of the Segre variety. To see this, let 
ei, . . . , be a basis of Ei and e^, . . . , e''^ the dual basis of E*. Assume, to avoid trivialities, 
that vi, V2 > ei. Given Ti e Vi^Ei we may write Ti = ui0ei + • • • + neii8'eei for some Uq S Vi. 
Similarly, given T2 G V2<8>-E5l may write Ti = wi(Sie^ + • • • + Wej^^e^'^ for some Wa G V2. Then 
Con{Ti(S)T2) = ui(S^wi H h nei^tf^ei- 

The graph used to define a set of tensor network states is often modeled to mimic the physical 
arrangement of the particles, with edges connecting nearby particles, as nearby particles are the 
ones likely to be entangled. 

Remark 1.2.2. The construction of tensor network states in the physics literature does not use 
a directed graph, because all vector spaces are Hilbert spaces, and thus self-dual. However the 
sets of tensors themselves do not depend on the Hilbert space structure of the vector space, 
which is why we omit this structure. The small price to pay is the edges of the graph must be 
oriented, but all orientations lead to the same set of tensor network states. 

1.3. Grasedyck's question. Lars Grasedyck asked: 

Is TNS{T, e, V) Zariski closed? That is, given a sequence of tensors T,. € V that converges 
to a tensor Tq, ifT^ E TNS{T,e,Y) for all e / 0, can we conclude Tq G rA^5(r,e, V) ? 

He mentioned that he could show this to be true when T was a tree, but did not know the 
answer when F is a triangle. 



v_l 




Definition 1.3.1. A dimension Vj is critical, resp. subcritical, resp. supercritical, if Vj = 
^see{j)^s, resp. Vj < n^ggj-j^e^, resp. Vj > n^geO)^^- If rA^S'(r,e, V) is critical for all j, we 
say TA^S'(r, e, V) is critical, and similarly for sub- and super-critical. 

Theorem 1.3.2. TNS(T, e, V) is not Zariski closed for any F containing a cycle whose vertices 
have non-subcritical dimensions. 

Notation. GL{V) denotes the group of invertible linear maps V V. GL(Vi) x • • • x GL(Vn) 
acts on Vi® • • • 14 by (51, • • • , 9n) • • • • f n = (51 wi)® ■ ■ ■ ® {gnVn)- (Here Vj e Vj and 
the action on a tensor that is a sum of rank one tensors is the sum of the actions on the 
rank one tensors.) Let End(y) denote the set of all linear maps V V. We adopt the 
convention that End(Vi) x • • • x End(T4) acts on Fi(8) ■ ■ ■ Vn hy {Z\, . . . , Z„) • v\® ■ ■ ■ ® Vn = 
{Z\V\)® • • • ® {ZnV„). Let qI{V) denote the Lie algebra of GL{V). It is naturally isomorphic 
to End(y) but it acts on Vi (8) • • • (8) via the Leibnitz rule: {Xi, . . . , Xn) • f 1 Cg) • • • Cg) = 
{XiVi)^V2^ ■ ■ ■ <E> Vn + vi^{X2V2)<E>vs^ ■ ■ ■ Vn + ■ ■ ■ vi(E) • • • (g) 'i;„_i(g)(X„,f„). (This is because 
elements of the Lie algebra should be thought of as derivatives of curves in the Lie group at the 
identity.) If X C F is a subset, X C V denotes its closure. This closure is the same whether 
one uses the Zariski closure, which is the common zero set of all polynomials vanishing on X, or 
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the Euclidean closure, where one fixes a metric compatible with the linear structure on V and 
takes the closure with respect to limits. 

1.4. Connections to the GCT program. The triangle case is especially interesting because 
we remark below that in the critical dimension case it corresponds to 

End{Vi) X End{V2) x Endiy^) ■ Mmu/te3,e2,ei , 

where, setting Vi = E^^E^, V2 = E^^Ei, and V^. = E2®El, Mmu/te3,e2,ei G Vi®V2®V^ is the 
matrix multiplication operator, that is, as a tensor, MMulteg,e2,ei = IdEi®IdE2®IdE^- In 0] a 
geometric complexity theory (GCT) study of MMult and its GLiVi) x GL(V2) x GLiV^) orbit 
closure is considered. One sets ei = 62 = ea = n and studies the geometry as n ^ 00. It is 
a toy case of the varieties introduced by Mulmuley and Sohoni [I2l [I3l [3] , letting S'^C^ denote 
the homogeneous polynomials of degree d on (C^)*, the varieties are GL^2 • det„ C S'^C" 

2 2 

and ■ ^"-»" perm^ C S^C" . Here det^ G S'^C" is the determinant, a homogeneous 

polynomial of degree n in variables, n > m, £ G S^C^, perm^ G S^C^ is the permanent and 
an inclusion C"^ C C" has been chosen. In [11] it was shown that Endi^^2 -detn 7^ GL^2 ■ det„, 
and determining the difference between these sets is a subject of current research. 

The critical loop case with e^ = 3 for all s is also related to the GCT program, as it corresponds 
to the multiplication of n matrices of size three. As a tensor, it may be thought of as a map 
{Xi, . . . ,Xn) ^ trace(Xi • • ■ Xn)- This sequence of functions, indexed by n, considered as a 
sequence of homogeneous polynomials of degree n on © • • • © V^, is complete for the class 
VPe of sequences of polynomials of small formula size, see [2]. 

Acknowledgments. We thank David Gross for providing background information, Lars Grasedyck 
for introducing us to tensor network states and posing the Zariski closure question, and the ref- 
eree for suggestions that have substantially improved the presentation of the paper. 



2. Critical loops 

Proposition 2.0.1. Letvi = e2e3,V2 = e3ei,V3 = e2ei. Then riVS'(A, (e2e3, eaCi, e2ei), ¥[(8)1/2(8)^3) 
consists of matrix multiplication and its degenerations (and their different expressions after 
clianges of bases), i.e., 

TNS{A, (e2e3,e3ei,e2ei),yi0y2^'tl5) = End{Vi) x End{V2) x EndiVs) ■ Me2,e3,ei- 

It has dimension e^e^ + e^el + e|ef — (e| + e| + ef — 1). 

More generally, ifV is a critical loop, TNS{T, (e^ei, eie2, . . . , e^—ie^ ),Vi'S> • • • © K) is End(Vi) X 
• • • X End(V„) • Mg, where Mg : Vi x ■ ■ ■ x Vn C is the matrix multiplication operator 
{Xi,.. .,Xn) ^ trace(Xi ■■■Xn). 

Proof. For the triangle case, a generic element Ti € E2®E^0Vi may be thought of as a linear 
isomorphism £/2©-E'3 Vi, identifying Vi as a space of e2 x e3-matrices, and similarly for 
V2,V3- Choosing bases e^" for E*, with dual basis e^^.s for Es, induces bases for Vi etc.. 
Let 1 < i < e2, 1 < a < e3, 1 < n < ei. Then 

con{Ti®T2(®T^) = x^©y^^©z," 



which is the matrix multiplication operator. The general case is similar. 



□ 
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Proposition 2.0.2. The Lie algebra of the stahiUzer of Afe„ei,eie2,.--,en-ie„ in GL{Vi) x ••• x 
GL{Vn) is the image of 5l{Ei) © • • • © s[(£'„) under the map 

ai © • • • © a„ ^{IdE^^ai, -aJi^IdE2,0, . . . , 0) + {0, IdEi'S>a2, -a^(8)/dE3, 0, • • • , 0) 
H h (-a^(g)/dEi,0, . . . ,0,ld£;„_i©a„). 

Here 5i{Ej) C 9i{Ej) denotes the traceless endomorphisms and T as a superscript denotes 
transpose (which is really just cosmetic). 

The proof is safely left to the reader. 

Large loops are referred to as "1-D systems with periodic boundary conditions" in the 
physics literature and are often used in simulations. By Proposition 12.0.21 for a critical loop, 

dim(rAf5(r, e, V)) = efe^H |-e^_]^e^ + e^ef - (ef H he^- 1), compared with the ambient 

space which has dimension ef • • • e^. For example, when = 2 for all j, dim(TA^S'(r, e, V)) = 
12n + 1, compared with dim V = 4". 

3. ZARISKI CLOSURE 

Theorem 3.0.3. Letvi = e2e3,V2 =e3ei,V3 = e2ei. TLeii TAfS'(A, (e2e3, eaOi, e2ei), Vi®V'2<8)V3) 
is not Zariski closed. More generally any TNS{T, e, V) where T contains a cycle with no suh- 
critical vertex is not Zariski closed. 

Proof. Were T(A) := TNS{A, (6263, 6301, 0261), Vi(g)V20V3) Zariski closed, it would be 

(2) GL{Vi) X GL{V2) X GLiVs) ■ Me^.ea.ei- 

To see this, note that the G = GLiVi) x GL(V2) x GL(V3) orbit of matrix multiplication is a 
Zariski open subset of T{A) of the same dimension as T(A). 

We need to find a curve g(t) = {gi{t), g2{t), §3(1)) such that gj{t) G GL{Vj) for all t / and 
limi^o5(*) • ,63, ei is both defined and not in End{Vi) x End{V2) x End{Vs) • Me2,e3.ei- 

Note that for [x, Y, Z) G GL{Vi) x GL{V2) x GL{V3), we have {X, Y, Z) ■ Me^.ea.ei (P, Q, R) = 
trace{X {P)Y{Q)Z{R)). Here X : E^^E^ E^^E^, Y : E^^Ei E^^Ei, Z : El®E2 
El®E2. 

Take subspaces C/S2-B3 C E'^fS'-Es, Ue-^^^e^ C £'3©£'i. Let UeiE2 '■= Con{UE2E3,UE3Ei) C 
E2®Ei be the images of all the pq G E2®Ei where p G UE2E3 and q G Ue-^Ei (i-e., the matrix 
multiplication of all pairs of elements). Take Xq,Yo,Zo respectively to be the projections to 
UE2E3, UesEi and UeiE2'^- Let X\^Y\,Z\ be the projections to complementary spaces (so, e.g., 
Xo + = Idv*). For P G Fi*, write Pq = Xq{P) and Pi = Xi(P), and similarly for Q, R. 

Take the curve [X^Yu Zt) with Xt = ^(Xq + tXi), Yt = ^(^0 + tYi), Zt = ^(Zq + tZi). 
Then the limiting tensor, as a map Vi x V2 x F3* — )• C, is 

(P, Q, P) 1-^ trace(PoQo-Ri) + trace(PoQiPo) + trace(PiQoPo)- 

Call this tensor M. First observe that M uses all the variables (i.e., considered as a linear 
map M : V]* — > V2©V3, it is injective, and similarly for its cyclic permutations). Thus it is 
either in the orbit of matrix multiplication or a point in the boundary that is not in Endiyi) x 
End{y2) X End{V3) ■ Me2,e3,ei) because all such boundary points have at least one such linear 
map non-injective. 

It remains to show that there exist M such that M ^ G ■ Me2,e3,ei To prove some M is a 
point in the boundary, we compute the Lie algebra of its stabilizer and show it has dimension 
greater than the the dimension of the stabilizer of matrix multiplication. One may take block 
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matrices, e.g., 

^0 =(l o) ' ^1 = (o *) ' 

and Yq, Yi have similar shape, but Zq, Z\ have the shapes reversed. Here one takes any sphtting 
ej = e^- + e^' to obtain the blocks. 

For another example, if one takes = e for all j, Xq, Iq) ^\ to be the diagonal matrices and 
and Xi, Yi, Zq to be the matrices with zero on the diagonal, then one obtains a stabilizer of 
dimension 4e^ — 2e > 3e^ — 1. (This example coincides with the previous one when all ej = 2.) 

To calculate the stabilizer of M, first write down the tensor expression of M G Vi(8'V2<8>V3 with 
respect to fixed bases of Vi, V2, V3. Then set an equation (X, y, Z).M = where X G 0t(Vi), 
Y S 0^(V2) and Z G 0t(V3) are unknowns. Recall that here the action of (X, y, Z) on M is the Lie 
algebra action, so we obtain a collection of linear equations. Finally we solve this collection of 
linear equations and count the dimension of the solution space. This dimension is the dimension 
of the stabilizer of M in GL{Vi) x GL{V2) x GL{V2,). 

To give an explicit example, let ei = e2 = 63 = e and let Xq = diag{x\^ ...^x'g), Yq = 
diag{yl...,y^), Zq = diag{zl z^), Xi = [x]) - Xq, ^ = (yj) - Yq, Zi = (zj) - Zq. Then 

e 

Let X = YI be an element of 0[(Vi), where {X.-. } is a basis of 0[(Vi), and define Y 

and Z in the same pattern with coefficients 6i 's and cl 's, respectively. Consider the equation 

U ) U > 

C) 0) {') 

{X, Y, Z).T = and we want to solve this equation for 's, bl and ci 's. For these equations 
to hold, the coefficients of z^'s must be zero. That is, for each pair of indices we have: 

e , J 

k,i=i ' ' ' ' » 

For these equations to hold, the coefficients of yl^s must be zero. For example, if s ^ j, r 7^ s 
then we have: 




Now coefficients of x terms must be zero, for instance, if i 7^ j and i ^ r, then we have: 

(') C) 

If one writes down and solves all such linear equations, the dimension of the solution is 4e^ — 2e. 

The same construction works for larger loops and cycles in larger graphs as it is essentially 
local - one just takes all other curves the constant curve equal to the identity. □ 

Remark 3.0.4. When ei = e2 = 63 = 2 we obtain a codimension one component of the boundary. 
In general, the dimension of the stabilizer is much larger than the dimension of G, so the orbit 
closures of these points do not give rise to codimension one components of the boundary. It 
remains an interesting problem to find the codimension one components of the boundary. 
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4. Algebraic geometry perspective 

For readers familiar with algebraic geometry, we recast the previous section in the language 
of algebraic geometry and put it in a larger context. This section also serves to motivate the 
proof of the previous section. 

To make the parallel with the GCT program clearer, we describe the Zariski closure as the 
cone over the (closure of) the image of the rational map (i.e., the "closure" of the map defined 
on a Zariski open subset) 

(3) FEnd{Vi) X FEnd{V2) x FEndiVs) P(Vi®y2®'V^3) 

([x],[y],[z])^(x,y,z).[Me,,e3,eJ. 

(Compare with the map ^/^ in [31 §7.2].) A dashed arrow is used to indicate the map is not 
everywhere defined. 

The indeterminacy locus (that is, points {[X], [Y], [Z]) where the map is not defined), consists 
of {[X],[Y],[Z]) such that for aU triples of matrices P,Q,R, tmce{X {P)Y (Q) Z {R)) = 0. In 
principle one can obtain ([2]) as the image of a map from a succession of blow-ups of ¥End{Vi) x 
FEnd(y2) X FEndiVs). (See, e.g., P p. 81] for the definition of a blow-up) 

One way to attain a point in the indeterminacy locus is to take ([^o]i [^]i [-^o]) as described 
in the proof. Taking a curve in G that limits to this point may or may not give something new. 
In the proof we gave two explicit choices that do give something new. 

A more invariant way to discuss that M ^ End{Vi) x End{V2) x End^V^) ■ Me2,e3,ei is to 
consider an auxiliary variety, called a subspace variety, 

Su&fi,...,f„(V) := {T eVi^---^Vn\ 3V^ C Vj,dimV^ = f,-, and T G y/(g) • • • V^,}, 

and observe that if T G Xj End{Vj) ■ Mg and T ^ XjGL{Vj) ■ Mg, then T £ 5n6f,„...,f„(V) where 
fj < ej for at least one j. 

The statement that "M uses all the variables" may be rephrased as saying that M ^ 

S'li6e2e3 — 1,6261 — 1,6361 — 1 

5. Reduction from the supercritical case to the critical case with the same 

GRAPH 

For a vector space W, let G{k, W) denote the Grassmannian of /c-planes through the origin in 
W. Let S — 7> G{k, W) denote the tautological rank k vector bundle whose fiber over E G G{k, W) 
is the fe-plane E. Assume ij < Vj for all j with at least one inequality strict. Form the vector 
bundle 5i(8> • • • (8) 5„ over G(fi, Vi) x • • • x G(f„, Vn), where Sj — >■ G(fj, Vj) are the tautological 
subspace bundles. Note that the total space of Si^ ■ ■ ■ ^ Sn maps to V with image Sub^(V). 
Define a fiber sub-bundle, whose fiber over (Ui x ••• x CZ„) G G(fi,Vi) x ••• x G(f„,V^) is 
TNS(r,e, Ui® ■■■<S) Un). Denote this bundle by TNS{T,e,Si® •••(g) 5„). 

The supercritical cases may be realized, in the language of Kempf, as a "collapsing of a 
bundle" over the critical cases as follows: 

Proposition 5.0.5. Assume ij := Ilggg^jjes < Vj. Then TNS{r,e,\') is the image of the 
bundle TNS{T, e , 5i (8) • • ■ (g) Sn) under the map to V. In particular 

n 

dim(riVS(r, e, V)) = dim(riV5(r, e, C*^i0 • • • (g) C*^")) + ^ ^j{^rj - fj). 

i=i 

Proof. If n^geQ-je^ < Vj, then any tensor T G ^■<8'((g'sem(j)-^s)'^('^ieo«t(i)-E't ); must lie in some 
V-(ii>{®s&n(j)Ps)®{®t^mit(j)P^t) with diniT^' = fj. The space TN S{r ,e,\) is the image of this 
subbundle under the map to V. □ 
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This type of bundle construction is standard, see [10\ I16| . Using the techniques in |16j . one 
may reduce questions about a supercritical case to the corresponding critical case. 

6. Reduction of cases with subcritical vertices of valence one 

The subcritical case in general can be understood in terms of projections of critical cases, but 
this is not useful for extracting information. However, if a subcritical vertex has valence one, 
one may simply reduce to a smaller graph as we now describe. 




Proposition 6.0.6. Let TNS{r, e , V) be a tensor network state, let v be a vertex ofF with 
valence one. Relabel the vertices such that v = vi and so that vi is attached by ei to V2- If 
vi < ei, then TNS{T, Vi0 • • • o = TNS{r, "e , Fi (g) V3 (g) ... (g K), where f is F with vi 
and ei removed, e is the vector (e2, e„) and Vi = Vi (S> V2. 

Proof. A general element in TNS{T, ,Vi(>^ ■ ■ ■ ® Vn) is of the form YlTj=i ^« '^iz ® Wz, where 
Wz (^V^®---® Vn. Obviously, TNS{T, ViO • • • ® K) ^ TNS{f, "e , (g) V3 ... K) =: 
TNS{r, e , V). Conversely, a general element in TNS{T, e , V)) is of the form X^j, Wz, 
Xz & Vi (S> V2. Since vi < ei, we may express Xz in the form YltLi ® ^i^; where ui, Uy^ is 
a basis of Vi. Therefore, rA^5(r, , V) D riV5'(f, "e , V).. □ 

7. Trees 

With trees one can apply the two reductions successively to reduce to a tower of bundles 
where the fiber in the last bundle is a linear space. The point is that a critical vertex is both 
sub- and supercritical, so one can reduce at valence one vertices iteratively. Here are a few 
examples in the special case of chains. The result is similar to the Allman- Rhodes reduction 
theorem for phylogenetic trees [1]. 

Example 7.0.7. Let T be a chain with 3 vertices. If it is supercritical, TA^S'(r, V) = 

Vi®V2®V^- otherwise TNS{T,'^,\) = Sufeei.eie^.ea (140^2(8) V3). 

Example 7.0.8. Let F be a chain with 4 vertices. If vi < ei and V4 < 03, then, writing 
W = Vi®V2 and U = V3(8)T4, by Proposition EHSl riVS'(r, ~i , V) is the set of rank at most 62 
elements in W®U (the secant variety of the two-factor Segre). Other chains of length four have 
similar complete descriptions. 

Example 7.0.9. Let T be a chain with 5 vertices. Assume that vi < ei, V5 < 64 and V1V2 > 62 
and V4V5 > 03. Then TNS{T, V) is the image of a bundle over G(e2, ^118)^2) x G(e3, V4(g)V5) 
whose fiber is the set of tensor network states associated to a chain of length three. 
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